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ABSTRACT 

Over the past years. Remote and Virtual Labs (RVLs) have gained increased attention for their potential to support 
technology-enhanced science education by enabling science teachers to improve their day-to-day science teaching. 
Therefore, many educational institutions and scientific organizations have invested efforts for providing online access to 
state-of-the-art science experiments via RVLs. Currently, there are existing initiatives for the storage and organization of 
existing RVLs into web-based repositories towards increasing their findability and enabling science teachers to search 
and retrieve them for further usage into their lesson plans. Nevertheless, most of these repositories adopt metadata models 
that store limited information related to the pedagogical context of their lesson plans. As a result, science teachers are not 
supported in selecting RVLs taking into consideration core elements of their lesson planning. In this paper, we aim to 
tackle this problem by proposing the ASK4Labs a web-based repository for supporting learning design driven RVLs 
recommendations. Preliminary evaluation results are also described, which indicate that the proposed recommender 
system can provide robust identification of appropriate RVLs based on the pedagogical context elements of the intended 
lesson plans. 
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1. INTRODUCTION 

Over the past years, technological advancements in the field of World Wide Web have allowed the 
advancement of physical laboratories and their partial replacement by remotely-operated labs (“remote labs”) 
and virtual labs (de Jong et al. 2013; Balamuralithara & Woods 2009). More specifically, remote labs provide 
students with the opportunity to collect data from a real physical laboratory, including real equipment from 
remote locations (Gomes & Bogosyan 2009). On the other hand, virtual labs represent interactive 
environments for designing and conducting simulated experiments (Balamuralithara & Woods 2009). Both, 
Remote and Virtual labs (RVLs) have gained increased attention for their potential to support technology- 
enhanced science education by enabling science teachers to improve their day-to-day science teaching. 
Additionally, it has been shown that RVLs are more effective in increasing sUidents’ interest in science and 
their engagement in related learning activities compared to Uaditional laboratories (Jaakkola et al. 2011; de 
Jong 2010; Kong et al. 2009). Thus, many educational institutions and scientific organizations have invested 
efforts for providing online access to state-of-the-art science experiments via RVLs (Gravier et al. 2008). 

Nevertheless, most RVLs currently available online are still scattered around the web. As a result, in 
order to increase their findability and enable science teachers to search and reUieve them for further usage, 
there are existing initiatives for their storage and organization into web-based repositories (Richter et al. 
2011; Maier & Niederstatter 2010). However, existing RVL repositories are adopting different metadata 
models for characterizing their RVLs. Furthermore, most of these repositories adopt metadata models that 
store limited information related to the pedagogical context of their lesson plans such as the teaching 
approach adopted, the subject domain, the intended educational objectives and the grade level. As a result, 
science teachers are not supported in selecting RVLs taking into consideration core elements of their lesson 
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planning. Within this context, the aim of this paper is to tackle this problem by proposing the ASK4Labs a 
web-based repository for supporting learning design driven RVLs recommendations. Moreover, preliminary 
evaluation results are described, which indicate that the proposed recommender system can provide robust 
identification of appropriate RVLs based on the pedagogical context elements of the intended lesson plans.. 

The paper is structured as follows. Following this introduction, section 2 reviews existing RVLs 
repositories and their metadata models and identifies their limitations related to the pedagogical information 
that they store. Section 3 presents the proposed ASK4Labs repository by presenting its main functionalities 
and the recommender system that it incorporates. Section 4 comprises the evaluation of the proposed 
recommender system. Finally, we discuss our main conclusions and ideas for further work. 


2. REVIEW OF EXISTING REPOSITORIES OF REMOTE AND 
VIRTUAL LABS 


According to Conole & Fill (2005) there are two (2) dimensions that constitute a learning design: 

■ The pedagogical context within which the learning design occurs. This includes, among others, the 
following elements: (a) the subject domain (i.e., physics, geography, math, arts, etc.), (b) the 
intended educational objectives (i.e., recall, understand, etc.), (c) the grade level (i.e. primary 
education, secondary education, higher education etc.) and (d) the teaching approach adopted (i.e., 
problem based learning, inquiry based learning, etc.). 

■ The learning activities undertaken to achieve the intended educational objectives. 

Thus, it is important that the pedagogical context elements are accommodated by the metadata model 
adopted by a RVL repository. This is essential in order to facilitate search and retrieval of RVLs taking into 
consideration elements of the pedagogical context of learning designs. 

In previous work, a review of existing repositories of remote and virtual labs was performed in order to 
highlight the metadata models adopted by existing RVLs repositories. Furthermore, we conducted a 
comparative analysis of the elements used by the metadata models of these repositories (Zervas et al. 2014). 
In the context of the present study, a meta-analysis of those results was performed in order to identify for 
each metadata model those metadata fields that store information related to the pedagogical context elements 
of a learning design. Table 1 presents the results of our findings. 

Table 1 . Overview of Existing RVLs Repositories and their Metadata Models 




Type of Labs 

No 

RVLs Repositories 

Remote 

Labs 

Virtual 

Labs 

1 

PhET 1 

- 

V 

2 

3 

Library of Labs 2 
Labshare 3 

V 

V 

4 

5 

Open Sources 
Physics 4 
Smart Science 5 

V 

V 

6 

Molecular 

Workbench 6 

- 

v 

7 

8 

Explore Learning 7 
ChemCollective s 

- 

V 

V 

9 

Remotely Controlled 
Laboratories (RCL) 9 


- 


Learning Design Pedagogical Context Elements 


Subject 

Domain 

Educational 

Objectives 

Grade 

Level 

Teaching 

Approach 

V 

V 

V 

- 

V 

- 

- 

- 

V 

- 

- 

- 

V 

- 

V 

- 

V 

V 

- 

- 

v 

- 

- 

- 

V 

V 

V 

_ 

V 

- 

- 

- 

_ 

_ 

_ 

_ 


1 http://phet.colorado.edu 

2 https://www.library-of-labs.org/ 
1 http://www.labshare.edu.au/ 

4 http://www.compadre.org/osp 

5 http://www.smartscience.net/ 

6 http://mw.concord.org/ 

7 http://www.explorelearning.com 
s http://www.chemcollective.org/ 
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10 

Skoool 10 

- 

V 

V 

- 

V 

- 

11 

iLabCentral 11 

V 

- 

V 

- 

V 

- 

12 

Lab2Go 12 

V 

V 

V 

- 

- 

- 

13 

WebLab Deusto 13 

V 

- 

V 

- 

- 

- 


As we can notice from Table 1, 12 (92,30%) of the examined repositories characterize their RVLs based 
on the subject domain, whereas only 5 (38,46%) characterize their RVLs based on the grade level. Moreover, 
only 3 (23,07) of the examined repositories adopt a metadata model that stores information about the 
educational objectives that their RVLs address. Finally, none of the examined repositories includes 
information about the teaching approach that their RVLs can be used. 

Following this analysis, we can identify that none of the examined repositories support all learning design 
pedagogical context elements. We consider this a major shortcoming in facilitating search and retrieval of 
RVLs based on their pedagogical characteristics. Within this context, our research problem is the design and 
evaluation of a recommender system that aims to support science teachers in selecting RVLs stored in a web 
repository taking into consideration pedagogical characteristics for given lesson plans. Our proposed solution 
is presented in this paper with the ASK4Labs web-based repository of RVLs that incorporates (a) an 
appropriately designed metadata model and (b) a recommender system that addresses this problem. 


3. THE ASK4LABS WEB-BASED REPOSITORY 

The ASK4Labs is a web-based repository that provides access to RVLs. It has been developed based on 
Drupal, which is a widely used, open source content management system and content management 
framework based on PHP and MySQL. For the purpose of our research, the ASK4Labs repository was 
populated with 45 RVLs. In the next paragraphs, we describe (a) the metadata model of the ASK4Labs 
repository, (b) its main functionalities and (c) the recommender system that incorporates the metadata model 
for facilitating selection and retrieval of appropriate RVLs based on the pedagogical context elements of 
given learning designs. 

3.1 Metadata Model 

The ASK4Labs repository adopts a metadata model that has been described in previous work (Zervas et al. 
2014). More specifically, the starting point for developing this metadata model was the outcomes of an 
extensive review of the metadata models of existing repositories of RVLs. Additionally, we considered for 
our model metadata elements that store information about the pedagogical context of a learning design, as 
described in section 2. Table 2 presents the metadata elements of the proposed metadata model. 


Table 2. ASK4Labs Repository Metadata Elements (Zervas et al. 2014) 


No 

Metadata 

Group 

Metadata 

Sub-Group 

Element Name 

Description 

Taxonomy 

Available? 

1 



Title 

Refers to the complete title of the lab 

No 

2 

3 

4 

General 

Metadata 

- 

Description 
Lab Type 

Language 

Provides a textual description of the lab 
Refers to the specific kind of the lab 
Refers to the languages that the lab is 
available in 

No 

YES 

YES 

5 



Keyword 

Refers to a set of terms that characterize the 
content of the lab 

No 

6 

7 

Organizational 

Metadata 

- 

Access Rights 
Rights Holder 

Refers to the lab’s access permissions 
Refers to those entities that hold the lab’s 

YES 

No 


9 http://rcl-munich.informatik.unibw-muenchen.de 

10 http://skoool.com 

1 1 http://ilabcentral.org 

12 http://www.lab2go.net 

13 https://www.weblab.deusto.es/weblab/client/#page=home 
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Lifecycle Date 

copyrights 

Refers to critical dates related to the lab’s 

No 

8 



lifecycle 

Provides information about contact details 



9 



Contact Details 

of the person or the organization responsible 
for the lab 

No 

10 



Cost 

Refers to any payment required for using the 
lab 

YES 

11 

12 



Licence 

Provider 

Provides information about copyrights and 
restrictions applied to the use of the lab. 
Provides information about the provider of 
the lab. 

Refers to each person (or entity) that has 

YES 

No 

13 



Contributor 

contributed in the making of the lab in its 
current state 

No 

14 



Version 

Provides information about the current 
version of the lab 

No 

15 



Status 

Provides information about the availability 
status of the lab 

YES 

16 


Learning 

Teaching Approach 

Refers to the teaching approach where the 
lab can be used 

No 

17 


Design 

Subject Domain 

Refers to the lab’s subject domain 

YES 

18 


Pedagogical 

Grade Level 

Refers to the grade level for which the lab 

YES 

Pedagogical 

Context 

can be used 

19 

Metadata 

Metadata 

Educational 

Refers to the educational objectives that the 

No 



Objectives 

lab addresses 

20 



Level of Difficulty 

Refers to the level of difficulty of the lab 

YES 

21 


- 

Intended End User 

Refers to the principal users for whom the 

YES 



Role 

lab was designed 


22 



Location URL 

Provides a URL for accessing the lab 

No 

23 

Technical 


Technical 

Refers to the technical requirements that are 

YES 

Metadata 


Requirements 

needed for using the lab. 

24 



Technical Format 

Refers to lab’s technical format. 

YES 

25 



Student’s Resource 

Refers to the type and the URL of student’s 
resource that is connected to the lab 

No 

26 

Content 

Metadata 

- 

Teacher’s Resource 

Provides the URL for accessing any lesson 
plan that can be used for exploiting the lab. 

No 

27 



Supportive App 

Provides the URL for accessing any 
supportive app that is connected to the lab. 

No 


3.2 Main Functionalities 

The ASK4Labs Repository targets the following user groups: 

■ RVLs owners, who want to characterize their RVLs with metadata and store them to the ASK4Labs 
repository, so as to increase their visibility and share them with science teachers for further usage 
into their day-to-day teaching activities. 

■ Science teachers, who want to search and find RVLs for using them into their lesson plans 
The main functionalities of ASK4Labs Repository can be summarized as follows: 

Store RVLs: RVLs owners are able to store in the ASK4Labs repository their RVLs along with metadata 
descriptions following the metadata model described in section 3.1. Figure 1 presents the process of storing a 
RVL to the repository by completing the appropriate metadata fields . 
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Complete 

General 

Metadata 




▼ Pedagogical 
Subject * 


Educational Level * 

Primary Education 

Upper Secondary Education 
Higher Education 


Difficulty * 

- Select a value - * 



Complete 

Pedagogical 

Metadata 


Show row weights 

Learning Objectives 

I *** 

Add another item 


Figure 1. Store a RVL to the ASK4Labs Repository by characterizing it with metadata 


Search for RVLs: Science teachers are able to search, browse and retrieve RVLs by using terms, which 
are matched with metadata descriptions of RVLs. Moreover, the searching mechanism incorporates a 
recommender system (it will be further described in section 3.3), which enables science teacher to receive 
recommendations about RVLs based on the pedagogical context elements of their lesson plans. Figure 2 
presents the process of searching RVLs in the ASK4Labs repository. 

View RVLs’ Metadata: Science teachers have the capability to view in details the metadata descriptions 
of RVLs, so as to be able to decide whether to use or not a specific RVL. Figure 3 presents the educational 
metadata of a selected RVL. 


ASK 4 Labs 

i epos limy of i 


& femote lobs 


Filter search 
results 


About S/jrch labs Contact 


Search results 


Lab Type 

• Virtual lab (33) 


Subject 

o Biology (5) 

► Astronomy (4) 

• Math (3) 

• Density (2) 

• Betaradtaoon(l) 

• 

• Electromagnetism (H 

• Boating and sinking (1) 
< Gamma radiation (1 1 


Online Labs List 


jravlty Force Lab 


Lab Type: Virtual Lab 
Subject: Physics. Motion 

Educational Level: Pnmaiy Education. Lower Secondary 
Education 

LanguagMtL English, Greek. Spanish. French. German 


Difficulty: Medium 
Keywords: Force. Gravity 

Cost: no 

Booking Required: no 

Description: Visualize the gravitational force that two objects exert on each other. Change 
properties of the objects in order to see how it changes the gravity force. 



Figure 2. Search RVLs Figure 3. View RVL Metadata and Provide Ratings / Comments 

Rate/Comment RVLs: Science teachers are able to provide their ratings and comments for the RVLs 
stored in ASK4Labs Repository. These ratings and comments can be related to the impressions of the science 
teachers who have used a specific RVL. Figure 3 presents the process of providing ratings/comments to a 
selected RVL. 

3.3 Recommender System 

The recommender system that has been incorporated in the ASK4Labs repository is a content-based 
recommender system that uses Vector Space Model (VSM) with basic TF-IDF (Term Frequency-Inverse 
Document Frequency) weighting (Lops et al. 2011). The recommender system aims to provide an ordered 
list of RVLs based on the learning design pedagogical context elements that are used during search by the 
science teachers. 
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More specifically, the pedagogical context of a learning design can be modelled as follows: 
Learning_Design l = { LD ,, LD 2 , LD 3 , LD 4 } where LD, represents the elements of the pedagogical context of a 
learning design. Additionally, for each element a different weighting factor can be applied namely Weight , = 
{ Wj, w 2 , tv ( , w 4 ) corresponding to each learning design vector space model elements. Different instantiations 
of the weight vector space model can be defined manually by each science teacher. 

On the other hand, a RVL can be modelled based on the different metadata elements already described in 
section 3.1, as follows: Metadata y = { MD h MD 2 , ...., MD 27 ), where MD, corresponds to each of the RVL 
metadata elements. However, not all metadata elements can be exploited in the context of the specific 
recommender system because not all of them can be mapped to the learning design vector space model 
elements. Thus, we performed a mapping between the learning design vector space model elements and the 
RVL metadata vector space model elements. Additionally, for the purpose of our research, we assigned 
percentages of relevancy for each metadata element that was mapped to the learning design pedagogical 
context elements. Table 3 presents this mapping and the assigned percentages of relevancy. 


Table 3. Learning Design Pedagogical Context Elements mapped to RVL Metadata 


Learning Design 
Pedagogical Context 
Elements 

Percentage of Relevancy 

RVL Metadata 

Subject Domain 

60% 

20% 

10% 

10% 

Subject Domain 

Keyword 

Title 

Description 


70% 

Learning Objectives 

Educational Objectives 

20% 

10% 

Keyword 

Description 

Grade Level 

80% 

20% 

Grade Level 
Description 

Teaching Approach 

70% 

30% 

Teaching Approach 
Description 


Based on the mapping presented in Table 3, the RVL metadata vector space model can be limited to 7 
distinctive elements as follows: Metadata y = { MD h MD 2 , MD 7 } where MDi=title, MD 2 =description, 
MD 3 =keyword, MD 4 =subject domain, MD 5 =Educational Objectives, MD 6 =grade level and MD 7 =teaching 
approach. 

According to the above modelling, next we present the process of generating the RVL recommendations 
in pseudo-code. As we can notice, the recommender system calculates an overall score for each RVL based 
on the query terms provided by the science teacher. It should be noted that we consider four query terms, 
each related to a separate Learning Design Vector Space Model element. Based on the calculated score, the 
RVLs are ranked from the most relevant to least relevant. 


For each query_term in LDi field { 

If (LDi = teaching approach) then { 


If 

query term found in MD 7 
Score+= Wj* tf-idf 

(query term. 

MD„, #RVLs) 

* 0,7 

If 

1 

query term found in MD 2 
Score+= Wj* tf-idf 

(query term. 

MD 3 , #RVLs) 

* 0,3 

If (LDi = grade level) then { 

If query term found in MD 6 
Score+= Wj* tf-idf 

(query term. 

MD 4 , #RVLs) 

* 0.8 

If 

query term found in MD 2 
Score+= Wj* tf-idf 

(query term. 

MD 3j #RVLs) 

* 0.2 


177 



ISBN: 978-989-8533-23-4 ©2014IADIS 


4. PRELIMINARY EVALUATION 

4.1 Method 

An experiment was performed towards the initial evaluation of the proposed recommender system. More 
specifically, the focus of the experiment was to evaluate the ranking accuracy of the recommender system 
utilizing the spearman’s rank correlation coefficient (Shani & Gunawardana, 2011). The evaluation 
methodology that was conducted included the following steps: 

Step 1: two queries were created representing two different pedagogical contexts for a learning design, as 
follows: Learning_Design 1 = {subject domain = "chemistry", educational objectives = "learn to carry out 
tests with chemical solutions", grade level = "lower secondary education", teaching approach = "inquiry 
based learning"), Learning_Design 2 = {subject domain = "physics", educational objectives = "learn about 
forces and balance", grade level = "primary education", teaching approach = "problem based learning"). 
Additionally, three instances of the weight vector space model were created, namely Weight j = {0.5, 0.2, 0.2, 
0.1} (emphasis given to the subject domain). Weight 2 = {0.2, 0.2, 0.5, 0.1) (emphasis given to the grade 
level). Weights = {0.2, 0.5, 0.1, 0.2} (emphasis given to the educational objectives). For both queries, each 
instance of the weight vector space model was applied and run at the ASK4Labs repository. Finally, the 
recommender system provided us with six ranked lists of RVLs. 

Step 2: we asked 30 secondary education science school teachers to validate the ranked lists produced 
from the recommender system. Each of the teachers was given the option to agree with the ranking or to 
propose his/her own ranked list. 

Step 3: Finally, we calculated the spearman’s rank correlation coefficient between the ranked lists 
produced by the recommender system and the ones provided by the science teachers. 

4.2 Results 

The preliminary evaluation results for ranking accuracy of the recommender system are presented in Figure 



Figure 4. Spearman’s Rank Correlation Coefficient per Ranked List (*= p<0,05, **=p<0,01) 

As we can notice from Figure 4, the spearman’s rank correlation coefficient was very high for all ranked 
lists produced by the recommender system. This provided us with evidence about the validity of our 
proposed mapping between leaning design elements and the RVL metadata elements, as well as about the 
usefulness of the percentages of relevancy for this mapping, as presented in Table 3. 

Furthermore, it is worth to notice that the ranking accuracy of the recommender system achieved the 
highest values for queries, where the emphasis to the weighting factors was given to those learning design 
pedagogical context elements that are mapped to RVL metadata modelled with taxonomies (see Table 2) 
compared to those modelled as free text elements. This can be explained by the fact that when metadata 
values are added as free text, polysemy and synonymy could be increased and the error margin of the 
recommender system could be also increased. 
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5. CONCLUSIONS AND FUTURE WORK 

In this paper, it was argued that there is a growing trend for the development of web-based repositories that 
provide access to RVLs. However, existing RVLs repositories are adopting different metadata models for 
characterizing their RVLs and these metadata models store limited information related to the core elements 
of lesson planning. Therefore, we presented the ASK4Labs, a web-based repository for supporting learning 
design driven RVLs recommendations. A preliminary evaluation was also performed, which focused on the 
ranking accuracy of the proposed recommender system. The results showed a high level of ranking accuracy 
especially when the weighting factors were assigned with emphasis given to learning design pedagogical 
context elements mapped to RVL metadata elements modeled with taxonomies. 

Future work includes additional evaluations of the ranking accuracy of the proposed recommender system 
with more users and with more search queries including different combinations of the weighting factors. 
Moreover, the recommender system could be enhanced by taking into account ratings of the users towards 
providing more accurate rankings of RVLs. 
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